Direct Feedback Alignment with Sparse Connections for Local Learning

Recent advances in deep neural networks (DNNs) owe their success to training algorithms that use backpropagation and gradient-descent. Backpropagation, while highly effective on von Neumann architectures, becomes inefficient when scaling to large networks. Commonly referred to as the weight transport problem, each neuron's dependence on the weights and errors located deeper in the network require exhaustive data movement which presents a key problem in enhancing the performance and energy-efficiency of machine-learning hardware. In this work, we propose a bio-plausible alternative to backpropagation drawing from advances in feedback alignment algorithms in which the error computation at a single synapse reduces to the product of three scalar values. Using a sparse feedback matrix, we show that a neuron needs only a fraction of the information previously used by the feedback alignment algorithms. Consequently, memory and compute can be partitioned and distributed whichever way produces the most efficient forward pass so long as a single error can be delivered to each neuron. Our results show orders of magnitude improvement in data movement and $2\times$ improvement in multiply-and-accumulate operations over backpropagation. Like previous work, we observe that any variant of feedback alignment suffers significant losses in classification accuracy on deep convolutional neural networks. By transferring trained convolutional layers and training the fully connected layers using direct feedback alignment, we demonstrate that direct feedback alignment can obtain results competitive with backpropagation. Furthermore, we observe that using an extremely sparse feedback matrix, rather than a dense one, results in a small accuracy drop while yielding hardware advantages. All the code and results are available under https://github.com/bcrafton/ssdfa.Comment: 15 pages, 8 figure

Direct Feedback Alignment With Sparse Connections for Local Learning

Recent advances in deep neural networks (DNNs) owe their success to training algorithms that use backpropagation and gradient-descent. Backpropagation, while highly effective on von Neumann architectures, becomes inefficient when scaling to large networks. Commonly referred to as the weight transport problem, each neuron's dependence on the weights and errors located deeper in the network require exhaustive data movement which presents a key problem in enhancing the performance and energy-efficiency of machine-learning hardware. In this work, we propose a bio-plausible alternative to backpropagation drawing from advances in feedback alignment algorithms in which the error computation at a single synapse reduces to the product of three scalar values. Using a sparse feedback matrix, we show that a neuron needs only a fraction of the information previously used by the feedback alignment algorithms. Consequently, memory and compute can be partitioned and distributed whichever way produces the most efficient forward pass so long as a single error can be delivered to each neuron. We evaluate our algorithm using standard datasets, including ImageNet, to address the concern of scaling to challenging problems. Our results show orders of magnitude improvement in data movement and 2× improvement in multiply-and-accumulate operations over backpropagation. Like previous work, we observe that any variant of feedback alignment suffers significant losses in classification accuracy on deep convolutional neural networks. By transferring trained convolutional layers and training the fully connected layers using direct feedback alignment, we demonstrate that direct feedback alignment can obtain results competitive with backpropagation. Furthermore, we observe that using an extremely sparse feedback matrix, rather than a dense one, results in a small accuracy drop while yielding hardware advantages. All the code and results are available under https://github.com/bcrafton/ssdfa

OmoMYC blunts promoter invasion by oncogenic MYC to inhibit gene expression characteristic of MYC-dependent tumors.

MYC genes have both essential roles during normal development and exert oncogenic functions during tumorigenesis. Expression of a dominant-negative allele of MYC, termed OmoMYC, can induce rapid tumor regression in mouse models with little toxicity for normal tissues. How OmoMYC discriminates between physiological and oncogenic functions of MYC is unclear. We have solved the crystal structure of OmoMYC and show that it forms a stable homodimer and as such recognizes DNA in the same manner as the MYC/MAX heterodimer. OmoMYC attenuates both MYC-dependent activation and repression by competing with MYC/MAX for binding to chromatin, effectively lowering MYC/MAX occupancy at its cognate binding sites. OmoMYC causes the largest decreases in promoter occupancy and changes in expression on genes that are invaded by oncogenic MYC levels. A signature of OmoMYC-regulated genes defines subgroups with high MYC levels in multiple tumor entities and identifies novel targets for the eradication of MYC-driven tumors

A Meta-Regression Analysis to Evaluate the Influence of Branched-Chain Amino Acids in Lactation Diets on Sow and Litter Growth Performance

A meta-regression analysis was conducted to evaluate the effects of branched-chain amino acids (BCAA) in lactating sow diets on litter growth performance, sow body weight change, and sow feed intake. Thirty-four publications that represented 43 trials with similar dietary Lys, but varying BCAA were used to develop a database that contained 167 observations. Diets from each trial were reformulated using NRC nutrient loading values in an Excel-based spreadsheet. Significant predictor variables within three optimum equations developed for litter ADG included the count of weaned pigs per litter, NE, SID Lys, CP, sow ADFI, Val:Lys, Ile:Lys, and Leu:Val. The equations suggest that the number of pigs weaned per litter and increasing NE, ADFI, SID Lys, and CP for sows are large, positive drivers for litter growth. Among the BCAA, the models for litter ADG indicate a positive influence of increasing Ile:Lys and Val:Lys and reducing Leu:Val on litter growth. For sow BW change, significant predictor variables within two competing models included litter size at 24 h, sow ADFI, Leu:Lys, and Ile+Val:Leu. The models suggest that litter size after cross-fostering at the start of lactation influences the predicted degree of sow BW change, and increased sow ADFI will improve, or minimize, BW change during lactation. Within the BCAA, the models indicate that increasing dietary Leu:Lys will minimize sow BW change during lactation. Lastly, the optimum equation for sow ADFI included Leu:Trp, SID Lys, NE, CP, and Leu:Lys as significant predictive variables. The model indicates that reducing Leu:Trp and increasing Leu:Lys will positively impact sow feed intake during lactation. Overall, the prediction equations suggest that BCAA play an important role in litter growth, sow BW change, and feed intake during lactation; however, the influence of BCAA on these criteria is much smaller than that of other dietary components such as NE, SID Lys, sow ADFI, and CP. These responses suggest that the three BCAA are essential for lactating sow and litter growth performance, but the predicted influence of Leu, Ile, and Val differ among litter ADG, sow weight loss, and sow daily feed intake

The DEEP2 Galaxy Redshift Survey: Design, Observations, Data Reduction, and Redshifts

We describe the design and data sample from the DEEP2 Galaxy Redshift Survey, the densest and largest precision-redshift survey of galaxies at z ~ 1 completed to date. The survey has conducted a comprehensive census of massive galaxies, their properties, environments, and large-scale structure down to absolute magnitude M_B = -20 at z ~ 1 via ~90 nights of observation on the DEIMOS spectrograph at Keck Observatory. DEEP2 covers an area of 2.8 deg^2 divided into four separate fields, observed to a limiting apparent magnitude of R_AB=24.1. Objects with z < 0.7 are rejected based on BRI photometry in three of the four DEEP2 fields, allowing galaxies with z > 0.7 to be targeted ~2.5 times more efficiently than in a purely magnitude-limited sample. Approximately sixty percent of eligible targets are chosen for spectroscopy, yielding nearly 53,000 spectra and more than 38,000 reliable redshift measurements. Most of the targets which fail to yield secure redshifts are blue objects that lie beyond z ~ 1.45. The DEIMOS 1200-line/mm grating used for the survey delivers high spectral resolution (R~6000), accurate and secure redshifts, and unique internal kinematic information. Extensive ancillary data are available in the DEEP2 fields, particularly in the Extended Groth Strip, which has evolved into one of the richest multiwavelength regions on the sky. DEEP2 surpasses other deep precision-redshift surveys at z ~ 1 in terms of galaxy numbers, redshift accuracy, sample number density, and amount of spectral information. We also provide an overview of the scientific highlights of the DEEP2 survey thus far. This paper is intended as a handbook for users of the DEEP2 Data Release 4, which includes all DEEP2 spectra and redshifts, as well as for the publicly-available DEEP2 DEIMOS data reduction pipelines. [Abridged]Comment: submitted to ApJS; data products available for download at http://deep.berkeley.edu/DR4

Effects of Anacetrapib in Patients with Atherosclerotic Vascular Disease

BACKGROUND: Patients with atherosclerotic vascular disease remain at high risk for cardiovascular events despite effective statin-based treatment of low-density lipoprotein (LDL) cholesterol levels. The inhibition of cholesteryl ester transfer protein (CETP) by anacetrapib reduces LDL cholesterol levels and increases high-density lipoprotein (HDL) cholesterol levels. However, trials of other CETP inhibitors have shown neutral or adverse effects on cardiovascular outcomes. METHODS: We conducted a randomized, double-blind, placebo-controlled trial involving 30,449 adults with atherosclerotic vascular disease who were receiving intensive atorvastatin therapy and who had a mean LDL cholesterol level of 61 mg per deciliter (1.58 mmol per liter), a mean non-HDL cholesterol level of 92 mg per deciliter (2.38 mmol per liter), and a mean HDL cholesterol level of 40 mg per deciliter (1.03 mmol per liter). The patients were assigned to receive either 100 mg of anacetrapib once daily (15,225 patients) or matching placebo (15,224 patients). The primary outcome was the first major coronary event, a composite of coronary death, myocardial infarction, or coronary revascularization. RESULTS: During the median follow-up period of 4.1 years, the primary outcome occurred in significantly fewer patients in the anacetrapib group than in the placebo group (1640 of 15,225 patients [10.8%] vs. 1803 of 15,224 patients [11.8%]; rate ratio, 0.91; 95% confidence interval, 0.85 to 0.97; P=0.004). The relative difference in risk was similar across multiple prespecified subgroups. At the trial midpoint, the mean level of HDL cholesterol was higher by 43 mg per deciliter (1.12 mmol per liter) in the anacetrapib group than in the placebo group (a relative difference of 104%), and the mean level of non-HDL cholesterol was lower by 17 mg per deciliter (0.44 mmol per liter), a relative difference of -18%. There were no significant between-group differences in the risk of death, cancer, or other serious adverse events. CONCLUSIONS: Among patients with atherosclerotic vascular disease who were receiving intensive statin therapy, the use of anacetrapib resulted in a lower incidence of major coronary events than the use of placebo. (Funded by Merck and others; Current Controlled Trials number, ISRCTN48678192 ; ClinicalTrials.gov number, NCT01252953 ; and EudraCT number, 2010-023467-18 .)

Through-Wall Single and Multiple Target Imaging Using MIMO Radar

The ability to perform target detection through walls and barriers is important for law enforcement, homeland security, and search and rescue teams. Multiple-input-multiple-output (MIMO) radar provides an improvement over traditional phased array radars for through-wall imaging. By transmitting independent waveforms from a transmit array to a receive array, an effective virtual array is created. This array has improved degrees of freedom over phased arrays and mono-static MIMO systems. This virtual array allows us to achieve the same effective aperture length as a phased array with a lower number of elements because the virtual array can be described as the convolution of transmit and receive array positions. In addition, data from multiple walls of the same room can be used to collect target information. If two walls are perpendicular to each other and the geometry of transmit and receive arrays is known, then data can be processed independently of each other. Since the geometry of the arrays is known, a target scene can be created where the two data sets overlap. The overlapped scene can then be processed so that image artifacts that do not correlate between the data sets can be excised. The result gives improved target detection, reduction in false alarms, robustness to noise, and robustness against errors such as improperly aligned antennas. This paper explores MIMO radar techniques for target detection and localization behind building walls and addresses different mitigation techniques, such as a singular value decomposition of wavelet transform method to improve localization and detection of targets. Together, these techniques demonstrate methods that show a reduction in size and complexity of traditional through-wall radar systems while still providing accurate detection and localization. The use of the range migration algorithm in single and multi-target scenarios is shown to provide adequate imaging of through the wall targets in near and far field. Also, a multi-view algorithm is used to provide improved target detection and localization by fusing together multiple wall views

Tailoring Nanoscale Morphology of Polymer: Fullerene Blends Using Electrostatic Field

To tailor the nanomorphology in polymer/fullerene blends, we study the effect of electrostatic field (E-field) on the solidification of poly(3-hexylthiophene-2, 5-diyl) (P3HT):[6,6]-phenyl-C61-butyric acid methyl ester (PC60BM) bulk heterojunction (BHJ). In addition to control; wet P3HT:PC60BM thin films were exposed to E-field of Van de Graaff (VDG) generator at three different directions—horizontal (H), tilted (T), and vertical (V)—relative to the plane of the substrate. Surface and bulk characterizations of the field-treated BHJs affirmed that fullerene molecules can easily penetrate the spaghetti-like P3HT and move up and down following the E-field. Using E-field treatment, we achieved favorable morphologies with efficient charge separation, transport, and collection. We improve; (1) the hole mobility values up to 19.4 × 10–4 ± 1.6 × 10–4 cm2 V–1 s–1 and (2) the power conversion efficiency (PCE) of conventional and inverted OPVs up to 2.58 ± 0.02% and 4.1 ± 0.40%, respectively. This E-field approach can serve as a new morphology-tuning technique, which is generally applicable to other polymer–fullerene systems.This document is the unedited Author’s version of a Submitted Work that was subsequently accepted for publication as Elshobaki, Moneim, Ryan Gebhardt, John Carr, William Lindemann, Wenjie Wang, Eric Grieser, Swaminathan Venkatesan, Evan Ngo, Ujjal Bhattacharjee, Joseph Strzalka, Zhang Jiang, Qiquan Qiao, Jacob Petrich, David Vaknin, and Sumit Chaudhary. "Tailoring nanoscale morphology of polymer: Fullerene blends using electrostatic field." ACS Applied Materials & Interfaces 9, no. 3 (2016): 2678-2685, copyright © American Chemical Society after peer review. To access the final edited and published work see doi: 10.1021/acsami.6b10870. Posted with permission.</p

Tailoring Nanoscale Morphology of Polymer:Fullerene Blends Using Electrostatic Field

To tailor the nanomorphology in polymer/fullerene blends, we study the effect of electrostatic field (E-field) on the solidification of poly­(3-hexylthiophene-2, 5-diyl) (P3HT):[6,6]-phenyl-C61-butyric acid methyl ester (PC₆₀BM) bulk heterojunction (BHJ). In addition to control; wet P3HT:PC₆₀BM thin films were exposed to E-field of Van de Graaff (VDG) generator at three different directionshorizontal (H), tilted (T), and vertical (V)relative to the plane of the substrate. Surface and bulk characterizations of the field-treated BHJs affirmed that fullerene molecules can easily penetrate the spaghetti-like P3HT and move up and down following the E-field. Using E-field treatment, we achieved favorable morphologies with efficient charge separation, transport, and collection. We improve; (1) the hole mobility values up to 19.4 × 10^–4 ± 1.6 × 10^–4 cm² V^–1 s^–1 and (2) the power conversion efficiency (PCE) of conventional and inverted OPVs up to 2.58 ± 0.02% and 4.1 ± 0.40%, respectively. This E-field approach can serve as a new morphology-tuning technique, which is generally applicable to other polymer–fullerene systems